Towards Domain Independent Why Text Segment Classification Based on Bag of Function Words

Increased attention has been focused on question answering (QA) technology as next generation search since it improves the usability of information acquisition from web. However, not much research has been conducted on “non-factoid-QA”, especially on Why Question Answering (Why-QA). In this paper, we introduce a machine learning approach to automatically construct a classifier with function words as features to perform Why Text Segments Classification (WTS classification) by using SVM. It is a process of detecting text segments describing “reasons-causes” and is a subtask of Why-QA mainly related to an answer extraction part. We argue that function words are a strong discriminator for WTS classification. Furthermore, since function words appear in almost all text segments regardless of the domain of the topic, it also enables construction of a domain independent classifier. The experimental results showed significant improvement over state-of-the-art results in terms of accuracy of WTS classification as well as domain independent capability.